Predictive Entropy Search for Efficient Global Optimization of Black-box Functions
نویسندگان
چکیده
We propose a novel information-theoretic approach for Bayesian optimization called Predictive Entropy Search (PES). At each iteration, PES selects the next evaluation point that maximizes the expected information gained with respect to the global maximum. PES codifies this intractable acquisition function in terms of the expected reduction in the differential entropy of the predictive distribution. This reformulation allows PES to obtain approximations that are both more accurate and efficient than other alternatives such as Entropy Search (ES). Furthermore, PES can easily perform a fully Bayesian treatment of the model hyperparameters while ES cannot. We evaluate PES in both synthetic and realworld applications, including optimization problems in machine learning, finance, biotechnology, and robotics. We show that the increased accuracy of PES leads to significant gains in optimization performance.
منابع مشابه
Parallel Predictive Entropy Search for Batch Global Optimization of Expensive Objective Functions
We develop parallel predictive entropy search (PPES), a novel algorithm for Bayesian optimization of expensive black-box objective functions. At each iteration, PPES aims to select a batch of points which will maximize the information gain about the global maximizer of the objective. Well known strategies exist for suggesting a single evaluation point based on previous observations, while far f...
متن کاملPredictive Entropy Search for Multi-objective Bayesian Optimization with Constraints
This work presents PESMOC, Predictive Entropy Search for Multi-objective Bayesian Optimization with Constraints, an information-based strategy for the simultaneous optimization of multiple expensive-to-evaluate black-box functions under the presence of several constraints. PESMOC can hence be used to solve a wide range of optimization problems. Iteratively, PESMOC chooses an input location on w...
متن کاملA General Framework for Constrained Bayesian Optimization using Information-based Search
We present an information-theoretic framework for solving global black-box optimization problems that also have black-box constraints. Of particular interest to us is to efficiently solve problems with decoupled constraints, in which subsets of the objective and constraint functions may be evaluated independently. For example, when the objective is evaluated on a CPU and the constraints are eva...
متن کاملA deterministic global optimization using smooth diagonal auxiliary functions
In many practical decision-making problems it happens that functions involved in optimization process are black-box with unknown analytical representations and hard to evaluate. In this paper, a global optimization problem is considered where both the goal function f(x) and its gradient f (x) are black-box functions. It is supposed that f (x) satisfies the Lipschitz condition over the search hy...
متن کاملA Meta-heuristic Algorithm for Global Numerical Optimization Problems inspired by Vortex in fluid physics
One of the most important issues in engineering is to find the optimal global points of the functions used. It is not easy to find such a point in some functions due to the reasons such as large number of dimensions or inability to derive them from the function. Also in engineering modeling, we do not have the relationships of many functions, but we can input and output them as a black box. The...
متن کامل